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PREFACE 


This  final  report  was  prepared  by  University  of  Florida, 
Center  of  Intelligent  Machines  and  Robotics,  Department  of 
Mechanical  Engineering,  Gainesville  FL  32611,  under  Contract  ID 
F08637-90-C-0024.  This  report  is  based  on  work  completed 
June  1994. 

This  report  describes  the  development  and  evaluation  of  a 
system  where  an  operator  can  control  a  John  Deere  690C  excavator 
remotely  while  accomplishing  RRR  tasks.  With  this  remote-control 
system,  an  operator  can  direct  all  functions  of  the  excavator 
(propulsion,  digging,  compaction,  leveling,  etc.)  from  a  control 
console  located  remotely  from  the  vehicle.  This  report  documents 
the  hardware  utilized  in  the  project  and  the  software  developed  for 
communication  between  the  computer  on-board  the  excavator  and  the 
control  console  computer. 

The  work  was  performed  between  October  1993  and  June  1994. 
The  project  officer  was  Mr  Allen  D.  Nease. 
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1.  INTRODUCTION 

The  objective  of  this  project  was  to  develop  a  system  such  that  an  operator  could  control 
a  John  Deere  690C  excavator  remotely  while  accomplishing  RRR  tasks.  With  this  remote-control 
system,  an  operator  is  able  to  direct  all  functions  of  the  excavator  (propulsion,  digging, 
compaction,  leveling,  etc.)  from  a  control  console  located  remotely  from  the  vehicle.  As  such, 
an  important  element  of  the  project  is  to  provide  the  operator  with  sufficient  sensory  feedback 
so  that  the  system  can  be  effectively  controlled. 

This  report  will  document  the  hardware  utilized  in  the  project  and  the  software  developed 
for  communication  between  the  computer  on-board  the  excavator  and  the  control  console 
computer.  A  video  of  the  system  demonstration  has  also  been  prepared  and  has  been  delivered 
to  the  Air  Force  technical  program  director  separately  from  this  report 

2.  APPROACH 

The  objective  of  developing  a  remote  control  system  was  divided  into  the  following  four 
components: 

1)  control  console  design  and  fabrication 

2)  communication  of  the  control  panel  status  to  the  on-board  computer 

3)  response  of  the  excavator  based  upon  the  panel  status 

4)  feedback  of  video  images  from  the  work  site  to  the  operator 
Each  of  these  four  areas  will  be  discussed  subsequently. 
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2.1  Operational  Assumptions 

When  the  project  was  begun,  two  assumptions  were  made  dealing  with  the  operational 
scenario.  First,  it  was  assumed  that  the  excavator  engine  would  be  started  and  thus  it  would  not 
be  necessary  to  perform  this  task  remotely.  Second,  the  on-board  computer  would  be  manually 
turned  on  and  the  interface  software  started.  If  the  excavator  engine  was  ever  stopped  during 
operation  (as  when  the  emergency  stop  button  is  pressed  on  the  remote  control  panel),  the 
operator  would  have  to  get  in  the  excavator,  restart  the  engine,  reboot  the  on-board  computer,  and 
finally  restart  the  interface  software.  Performing  these  tasks  remotely  as  opposed  to  manually 
would  have  added  cost  to  the  project,  with  little  added  performance. 

2.2  System  Overview 

Two  computers  are  used  in  the  system  to  establish  open  loop  control  of  the  remote 
excavator.  One  computer  is  located  on-board  the  excavator  and  one  is  situated  with  the  control 
console  (see  Figures  1  and  2).  These  computers  handle  communications,  I/O,  and  the  required 
computations.  Specifically,  the  operator  controls  the  excavator  by  manipulating  the  joysticks  and 
switches  on  the  control  panel  (the  control  panel  is  a  subsystem  of  the  control  console).  The 
control  console  computer  constantly  determines  the  status  of  all  the  devices  and  then 
communicates  this  information  to  the  computer  on-board  the  excavator.  The  excavator  computer 
determines  how  the  vehicle  should  respond  based  on  the  current  control  panel  status  and  issues 
appropriate  commands.  For  example,  the  panel  status  might  show  that  the  operator  is  pressing 
the  left  propel  joystick  forward.  This  fact  is  communicated  to  the  on-board  computer.  This 
computer  realizes  that  the  hydraulic  valve  which  controls  the  left  wheels  of  the  excavator  must 
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be  opened  more  fully.  An  appropriate  analog  signal  is  generated  to  accomplish  this  task. 

No  low  level  feedback  is  communicated  back  to  the  operator.  That  is,  no  sensed  joint 
angle  positions,  vehicle  velocity  measurements,  etc.  are  returned  to  the  operator.  (It  should  be 
noted,  however,  that  the  communications  scheme  employed  would  allow  for  this  reverse  flow  of 
information.)  All  feedback  is  accomplished  via  two  video  cameras  which  are  mounted  on  the 
vehicle.  One  of  the  cameras  is  positioned  on  top  of  a  pan/tilt  mechanism  which  the  operator  can 
control  remotely  from  the  control  console.  The  operator  must  drive  the  vehicle,  and  perform  all 
crater  repair  tasks  based  solely  on  the  images  returned  by  the  video  system. 

2.3  Control  Console  Design  and  Fabrication 

The  tasks  which  the  operator  can  perform  from  the  control  console  may  be  divided  into 
the  following  three  categories: 

•  Mobility  functions.  The  operator  is  able  to  control  the  left  and  right  wheel  propel 
functions  as  well  as  the  engine  throttle. 

•  Action  functions.  These  functions  deal  with  the  control  of  the  four  arm  joint 
actuators,  the  leveling  blade,  tool  trailer,  and  tool  latching  pin. 

•  Supervisory  functions.  These  functions  allow  the  operator  to  control  the  two 
video  cameras  and  the  one  pan/tilt  mechanism. 

A  drawing  of  the  control  console  is  shown  in  Figure  3.  As  shown  in  the  figure,  the  console 
consists  of  a  pair  of  racks  in  which  a  rack  mountable  computer,  a  TV  monitor,  and  a  control 
panel  are  mounted.  The  TV  monitor  displays  the  video  scenario  at  the  operating  site.  The 
control  panel  has  7  momentary  three-position  focker  switches,  one  on-off  rocker  switch,  two 
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single-axis  joysticks,  and  two  dual-axis  joysticks  to  control  the  excavator.  A  drawing  of  the 
layout  of  the  control  panel  is  shown  in  Figure  4. 

An  Industrial  Computer  Source  ICS-286  rack  mountable  computer  is  mounted  in  the  rack 
of  the  console.  One  digital  I/O,  CI0-DI024  from  Cyber  Research,  and  one  DT2821  board  are 
installed  in  the  console  computer.  The  digital  I/O  board,  with  the  base  address  of  0x300,  reads 
discrete  signals  of  switching  actions.  The  switches  and  button  receive  the  necessary  +5  VDC 
power  from  the  ICS-286  computer  through  the  CI0-DI024  digital  I/O  interface.  A  DT2821 
board,  with  the  base  address  0x240,  reads  the  analog  outputs  from  the  joysticks.  A/D  channels 
0,  2,  4,  and  6  are  connected  to  the  dual-axis  joysticks  to  actuate  swing,  boom,  arm,  and  bucket, 
respectively.  A/D  channels  8  and  10  are  connected  to  the  single-axis  joysticks,  respectively,  to 
drive  the  left  and  right  propels.  Analog  signals  are  single-ended  connected  to  the  terminal  strip. 

It  is  important  to  note  that  the  A/D  channels  of  the  DT2821  can  only  take  up  to  ±10 
VDC.  The  output  voltages  from  the  joysticks  must  therefore  be  adjusted  to  fit  into  this  range. 
The  potentiometers  of  the  dual-axis  joysticks  turn  only  a  fraction  of  their  full  ranges  for  the 
travelling  angles.  Thus,  they  work  well  with  the  15  VDC  power  supplied  by  the  DT2821 
interface.  The  travelling  angles  for  the  single-axis  joysticks  almost  span  the  full  range  of  their 
attached  potentiometers,  however.  A  trimming  potentiometer  is  used  adjust  the  voltage  outputs 
to  within  +10  VDC. 

As  previously  stated,  the  switches  and  the  button  take  +5  VDC  from  the  CI0-DI024.  All 
the  switches  on  the  control  panel  are  normally  open  except  for  the  emergency  button  and  the 
two-position  rocker  switch.  To  avoid  drawing  too  much  current  when  any  of  the  switches  is 
closed,  a  500  ohm  resistor  is  used  with  each  of  the  switches  and  the  button.  A  resistor  with 
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Figure  4:  Layout  of  Control  Panel 
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larger  resistance  would  draw  less  current  from  the  computer  power  supply.  The  limit  of  the 
value  of  the  resistor  which  can  be  used  is  determined  by  the  sensitivity  of  the  digital  I/O 
interface.  The  computer  could  not  detect  the  changed  status  of  a  switch  if  too  little  current  was 
drawn  by  the  I/O  channels.  The  schematic  diagram  of  the  wiring  for  the  control  panel  is  shown 
in  Figure  5. 

The  joysticks  on  the  control  panel  duplicate  the  functions  of  the  control  levers  for  the  arm 
and  the  foot  pedals  for  driving  the  excavator.  This  facilitates  an  experienced  operator  of  the 
excavator  while  manipulating  the  teleoperation  system.  Proportional  control  is  used  for  all  the 
joystick  controls. 

The  two-position  rocker  switch  was  installed  to  toggle  between  two  cameras.  It  was  not 
used  in  the  project,  although  the  wiring  is  operational  and  toggling  the  switch  is  detected  by  the 
console  computer.  Similarly,  pressing  of  the  emergency  button  is  detected  by  the  console 
computer,  although  a  separate  kill  button  was  used  for  the  excavator  in  place  of  this  button  on 
the  panel. 

The  other  momentary  three-position  switches  can  be  grouped  into  two  sets:  those  for  the 
action  functions  and  those  for  the  video  system.  On  the  left-hand  side  of  the  control  panel,  there 
is  (a)  one  switch  used  for  pushing  in  or  out  the  tool  latching  pin  at  the  end  of  boom,  (b)  one  for 
trailer  up  and  trailer  down,  and  (c)  one  for  blade  up  and  down.  On  the  right-hand  side  of  the 
control  panel,  there  is  (a)  one  switch  used  for  panning  camera  A  left  and  right,  (b)  one  for  tildng 
camera  A  up  and  down,  and  (c)  one  for  zooming  in  and  out  for  camera  A.  The  technical  manual 
for  the  cameras  has  been  ordered,  but  has  yet  to  be  received.  As  such,  the  zooming  function  has 
not  been  implemented,  although  the  console  computer  does  detect  when  the  switch  is  pressed. 
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Figure  5:  Wiring  Diagram  for  Control  Panel 


% 


2.4  Excavator  Control 


A  KMS-386  industrial  computer  is  used  in  the  excavator.  The  KMS-386  requires  +24 
VDC  power.  The  current  status  of  the  control  panel  (the  binary  status  of  every  button  and  switch 
and  the  analogue  status  of  each  joystick)  is  communicated  from  the  control  console  to  the  on¬ 
board  computer  via  a  serial  communications  link  (sec  Section  2.5).  The  KMS-386  computer 
must  control  the  various  actuators  on  the  excavator  in  response  to  the  current  panel  status. 

Three  Data  Translation  DT2821  I/O  interfaces  and  one  Metrabyte  DDA-06  board  are 
installed  in  the  on-board  computer.  Although  each  DT2821  is  capable  of  doing  8  differential 
A/D  conversions  or  16  single-ended  A/D  conversions,  2  D/A  conversions,  and  16  digital  I/O, 
only  six  D/A  channels  are  required  for  this  stage  of  the  project  A/D  and  D/A  conversions  both 
have  12-bit  resolution.  The  first  two  DT2821  cards,  with  base  addresses  of  0x240  and  0x280, 
respectively,  are  used  to  control  the  motions  of  the  swing,  arm,  boom,  and  bucket  The  D/A 
channels  of  the  third  DT2821  card,  with  base  address  of  0x2C0,  are  used  to  drive  the  left  and 
right  propels.  Each  DT2821  card  is  connected  to  a  DT707  data  terminal  board  to  facilitate  wiring 
(see  Figure  6). 

A  digital  I/O  board,  DDA-06  from  Metrabyte,  is  used  to  control  the  pan/tilt  mechanism 
used  with  camera  A.  The  DDA-06  sends  out  discrete  signals  to  an  ERB-24  relay  board  to 
actuate  the  motions  according  to  the  instructions  from  the  control  panel.  The  base  address  of  the 
DDA-06  is  0x300.  Since  the  ERB-24  needs  110  VAC  power,  it  is  connected  to  a  24  VDC/HO 
VAC  invertor.  The  schematic  diagram  for  the  wiring  of  the  ERB-24  is  shown  as  Figure  7. 
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Figure  6:  Wiring  diagram  for  DT707  Data  Terminal 
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Figure  7:  Schematic  Diagram  for  the  Wiring  of  the  ERB-24  Relay  Board 
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2.5  Communications 

Serial  communications  was  established  between  the  ICS-286  computer  (console  computer) 
and  the  KMS-386  computer  (on-board  computer).  The  COMl  port  was  used  on  each  of  the 
computers.  The  original  proposal  called  for  radio  communication  between  the  computers  via  a 
pair  of  radio  modems  at  a  speed  of  1200  baud. 

The  modems  were  successfully  installed  and  tested  in  the  QMAR  laboratory.  However, 
due  to  a  lack  of  frequency  clearance,  the  radio  modems  could  not  be  used  at  Tyndall  Air  Force 
Base.  It  was  proposed  that  the  excavator  be  tethered.  Normal  serial  communications  via  wire, 
however,  is  designed  for  a  distance  of  50  ft.  A  pair  of  Inmac  data  drivers  were  used  to  increase 
the  allowable  length  of  the  communications  tether.  Data  drivers  can  reach  3.3  miles  at  9600 
baud  with  24  gauge  wires. 

Each  data  driver  uses  two  pairs  of  wires  for  communication,  one  pair  for  transmitting  data 
and  another  pair  for  receiving  data.  The  control  sequence  is  identical  to  that  used  for  regular  RS- 
232C  serial  communication.  Although  it  is  a  bit  cumbersome  to  have  communication  through 
tethered  cables,  the  communication  speed  of  9600  baud,  was  eight  times  faster  than  could  be 
achieved  with  the  radio  modems. 

The  communications  software,  coded  in  Turbo  C,  was  created  with  hardware  interrupt 
driven  functions  so  that  a  tight  communication  loop  could  be  established.  The  communication 
sequence  starts  with  the  program  "CONTROL"  on  the  excavator  side.  When  the  program 
"CONTROL"  is  executed,  it  waits  for  a  prompt  to  synchronize  the  communications.  At  this 
point,  the  operator  can  start  the  program  "PANEL"  on  the  console  computer.  It  sends  out  a 
synchronization  prompt,  and  then  waits  for  another  synchronization  prompt  to  return  from  the 
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excavator.  After  the  synchronization  process  has  completed,  a  message  "Synchronized..."  is 
displayed  on  the  control  console  monitor  indicating  that  proper  conununication  is  now 
established.  Program  "PANEL"  then  reads  the  status  of  all  switches,  buttons,  and  joysticks  and 
sends  this  information  to  the  on-board  computer. 

Sixteen  bytes  of  data  are  passed  for  each  transmission  cycle  (see  Figure  8).  The  first  byte 
is  currendy  used  to  stop  the  control  sequence,  although  its  function  can  be  expanded  to  instruct 
multiple  vehicles.  The  next  three  bytes  arc  used  to  control  the  relays  on  the  ERB-24  which 
control  the  pan/tilt  mechanism  for  camera  A.  The  remainder  of  the  bytes  are  used  to  indicate 
the  status  of  the  switches,  buttons,  and  joysticks.  A  time-out  feature  is  also  incorporated  in  the 
software  in  order  to  halt  the  excavator  if  the  panel  status  has  not  been  communicated  from  the 
console  computer  within  a  specified  time  interval.  The  communications  programs  have  been 
delivered  to  Tyndall  Air  Force  Base  separate  from  this  report. 

2.6  Video  Feedback  System 

Two  SONY  DXC-325  CCD  color  video  cameras  are  used  to  send  video  images  to  the 
control  console  so  that  the  operator  can  view  the  scene  at  the  remote  site.  Camera  A  is  mounted 
on  a  pan/tilt  mechanism  on  the  top  of  cab.  The  pan/tilt  mechanism  can  tilt  up  15°  and  down 
60°  at  a  speed  of  3.0°  per  second.  It  can  also  pan  through  a  range  of  330o  at  a  speed  of  12° 
per  second.  A  wiring  diagram  for  the  pan/tilt  mechanism  is  shown  in  Figure  9.  Camera  B  is 
mounted  on  the  side  the  excavator  and  is  aimed  toward  the  arm  in  order  to  aid  the  operator  in 
digging  operations.  Both  cameras  require  -f  12  VDC  from  an  invertor. 

Originally,  the  video  signals  from  the  cahieras  were  to  be  sent  to  the  control  console  by 
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Figure  9:  Wiring  Diagram  for  the  Pan/Tilt  Mechanism 
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means  of  a  video  transmitter^and  receiver.  However,  since  frequency  clearance  could  not  be 
obtained,  the  video  signals  from  the  cameras  were  sent  to  the  console  through  a  pair  of  coaxial 
cables.  Both  cables  connect  directly  to  the  console  monitor.  The  operator  can  select  the  desired 
camera  by  pressing  a  selector  switch  on  the  monitor  itself. 

3.  SYSTEM  DEMONSTRATION 

On  June  4, 1991,  a  successful  demonstration  took  place  at  Tyndall  Air  Force  Base.  All 
the  designed  functions  of  the  added  system  for  the  JD690C  excavator  worked  properly.  Although 
maneuvering  the  remote  excavator  was  not  a  simple  task  for  an  inexperienced  operator,  an 
experienced  operator  was  able  to  efficiently  maneuver  the  excavator,  and  perform  a  variety  of 
crater  repair  tasks. 

4.  CONCLUSION  AND  RECOMMENDATIONS 

Certain  modifications  can  be  made  to  the  system  to  improve  its  performance.  The 

following  comments/recommendations  are  made: 

1)  The  maximum  flow  capacity  of  the  hydraulic  system  of  the  JD690C  is  apparently  not 
large  enough  to  support  the  full  simultaneous  operation  of  the  arm  actuators.  Drifting  of  the  arms 
has  been  observed  when  more  than  one  joint  is  moved.  In  the  developed  system,  the  trailer  is 
able  to  be  lifted  by  the  switching  action,  but  the  hydraulic  system  is  not  powerful  enough  to  hold 
it  in  place.  A  solution  to  these  problems  is  to  carefully  calculate  the  required  flow  volume  at  the 
vehicle  design  stage  in  order  to  cover  the  full  extent  of  system  operation.  It  would  be  very 
difficult  and  expensive  to  modify  the  existing  excavator,  however.  (It  should  be  noted  that  the 
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problem  with  the  availability^ of  fluid  flow  is  not  due  in  any  part  to  this  effort  of  developing  a 
remote  operator’s  console.  The  same  problem  exists  when  an  operator  is  in  the  cab.) 

2)  When  the  operator  is  driving  the  excavator,  the  only  feedback  is  the  images  provided 
by  the  two  video  cameras  on  the  excavator.  The  cameras  offer  a  limited  view  of  the  remote  site. 
The  video  system  also  falls  short  in  conveying  the  locadon  of  the  excavator  in  world  coordinates. 
It  could  be  possible  for  the  operator  to  become  disoriented.  An  improvement  to  the  system 
would  result  if  global  excavator  coordinates  were  known  and  displayed  for  the  operator  on  a  top 
down  view  map  of  the  environment  Global  excavator  coordinates  are  not  currently  determined, 
but  future  efforts  in  navigadon  will  provide  this  data. 

3)  Audio  feedback  from  the  operating  site  can  also  be  helpful  to  the  operator  in 
understanding  the  situadon.  Engine  noise,  etc.  provides  a  good  indicadon  of  the  situation  at  the 
remote  site. 

4)  It  is  not  economical  or  practical  to  install  cameras  all  around  the  vehicle.  Yet  the 
operator  may  require  a  "bird’s-eye-view"  in  order  to  visualize  the  excavator  status  (blade 
up/down,  arm  configuration,  tool  trailer  up/down,  etc.)  A  graphic  display  of  the  remote  vehicle 
which  uses  the  feedback  joint  angles,  trailer  position,  etc.  and  the  global  coordinates  of  the 
vehicle  in  the  working  environment  would  be  helpful. 

5)  In  the  current  system,  there  is  no  feedback  from  the  pan/tilt  mechanism.  It  is  not  easy 
for  the  operator  to  orient  the  view  presented  on  the  monitor  with  respect  to  the  excavator. 

6)  The  throttle  control  will  be  better  if  there  is  feedback  to  show  the  position  of  the  linear 
actuator. 

7)  Tethered  cables  could  impede  the  movement  of  the  excavator.  Radio  communication 
should  be  used  for  the  runway  repair  vehicle. 
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